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ABSTRACT 

At  present,  there  are  no  standards  for  assessing  the  value  of  intangible  assets  or 
intellectual  capital.  Historically,  a  number  of  frameworks  have  evolved,  each  with  a 
different  focus  and  a  different  assessment  methodology.  In  order  to  assess  that 
knowledge  management  initiatives  contributed  to  the  fight  against  terrorism  in  Canada,  a 
results-based  framework  was  selected,  customized  and  applied  to  CRTI  (a  networked 
science  and  technology  program  to  counter  terrorism  threats).  This  chapter  describes  the 
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step  by  step  process  of  how  the  results-based  framework  was  applied  to  measure  the 
value  contributed  by  knowledge-based  assets.  A  combination  of  qualitative,  quantitative 
and  anecdotal  assessment  techniques  was  used  and  a  map  was  employed  to  visualize  the 
evaluation  results.  The  strengths  and  weaknesses  of  this  particular  approach  are  discussed 
and  specific  examples  from  CRTI  are  presented  to  illustrate  how  other  organizations  can 
use  this  method  to  assess  the  value-added  to  innovation  and  research  and  development 
using  a  results-based  framework. 


INTRODUCTION 

Executives  and  managers  would  be  hard-pressed  to  argue  against  the  theoretical 
foundations,  goals  and  intended  results  of  the  discipline  of  Knowledge  Management 
(KM)11  as  a  potential  benefit  to  their  organizations.  It  is  self-evident,  even  trite,  to  state 
that  a  successful  organization  must  manage  its  intellectual  capital  well  to  achieve 
competitive  advantage,  be  more  innovative  and  enhance  its  value.  The  challenge  lies  not 
in  accepting  these  maxims  but  in  the  practical  application  of  knowledge  management 
activities  and  subsequently  to  be  able  to  demonstrate  whether  these  same  activities 
actually  do  contribute  to  the  enhancement  of  the  organization  and  its  goals. 

This  was  the  challenge  faced  by  Defence  Research  and  Development  Canada’s  Centre  for 
Security  Science.  When  its  flagship  program,  the  Chemical,  Biological,  Radiological, 
Nuclear  and  Explosives  (CBRNE)  Research  and  Technology  Initiative  (CRTI)111  was 
originally  created  in  2002  in  response  to  the  changing  North  American  domestic  security 
environment,  the  need  for  a  KM  approach  was  obvious.  This  Canadian  Government 
program  intended  to  bring  together  multiple  federal  departments  and  agencies  to 
strengthen  a  national  science  and  technology  (S&T)  capability  to  counter  the  terrorist 
threats  from  chemical,  biological  and  radiological  or  nuclear  agents.  Only  a  KM 
approach  would  be  able  to  build  communities,  collaboration,  knowledge  sharing  and 
creation  where  such  a  domain  had  not  existed  previously.  A  robust  KM  program  was 


created  through  consultation  with  stakeholders  and  the  program  thrived,  becoming  a 
model  for  others. 

Yet,  did  the  KM  program  contribute  to  the  success  of  the  program?  Did  it  impact  the 
federal  laboratories’  capability  and  capacity  to  respond  to  CBRN  incidents  or  contribute 
to  focused  expertise,  knowledge  and  capabilities  of  Canadian  CBRN  S&T  performers  in 
the  short-term?  Did  it  assist  in  any  way  in  engaging  the  Canadian  Innovation  System  in 
CBRN  counter-terrorism  or  help  the  creation  of  industrial  products,  technologies  and 
knowledge  for  CBRN  counter-measures  in  the  medium  term?  Finally,  would  the  KM 
program  contribute  to  the  long-term  goals  of  building  the  Canadian  S&T  capacity  and 
capability  to  prepare  for,  prevent  and  respond  to  CBRN  attacks,  or  enhance  the 
communication,  cooperation,  collaboration,  and  interoperability  amongst  Canadian  and 
international  CBRN  counter-terrorism  communities,  or  eventually  to  a  effectively 
positioned  Canadian  S&T  innovation  system  that  contributed  to  national  and  international 
security? 

In  order  to  answer  these  questions  at  some  stage,  it  would  be  necessary  to  develop  a 
meaningful  measurement  tool  and  process  that  would  provide  qualitative  data  that  would 
be  useful  during  an  evaluation.  Measurement  tools  often  tend  toward  the  quantitative 
side,  measuring  occurrences  of  activities.  This  is  helpful  to  detennine  pull  on  services, 
increased  (or  decreased)  need  for  resources  and  to  measure  trends.  But  can  such 
measurement  processes  indicate  whether  the  KM  activities  themselves  actually  contribute 
to  the  attainment  of  organizational  objectives?  The  question  for  the  KM  Team  was:  do 
the  KM  activities  contribute  in  a  meaningful  way  to  the  mission  and  outcomes  of  the 
CRTI  and  can  that  be  measured? 

This  chapter  focuses  on  the  search  for  a  method  of  meaningful  value  measurement.  The 
main  goal  will  be  to  illustrate  the  use  of  a  result-based  management  accountability 
framework  (RMAF)  as  a  tool  to  measure  the  impact  of  knowledge  management  activities 
on  the  intangible  assets  of  an  S&T  counter-terrorism  organization.  The  key  alignment  of 
the  measurement  framework  and  the  strategic  importance  of  the  knowledge-based  assets 


will  be  presented  as  a  recommended  best  practice.  The  case  study  will  also  describe  the 
general  approach  to  identifying  the  intangible  assets  and  investigating  and  selecting  the 
best  approach  to  assessing  these  assets. 


BACKGROUND 

In  the  early  2000s  the  CRTI  was  born  within  a  cultural  milieu  that  recognized  the  need 
for  breaking  down  knowledge  stovepipes  and  the  advantages  of  working  collaboratively 
for  common  aims.  KM  authors  were  expounding  the  virtues  of  collaboration  and  the  need 
leveraging  knowledge  in  order  to  gain  the  “knowledge  advantage.”  (Prusak,  1996)  The 
Government’s  Standing  Committee  on  Industry,  Science  and  Technology  had  just 
released  A  Canadian  Innovation  Agenda  for  the  Twenty-First  Century 1V  in  which  it 
indicated  the  need  for  “more  coordination  of  intramural  S&T  activities  among  federal 
agencies,  as  well  as  greater  collaboration  on  major  horizontal  issues  —  those  that  cut 
across  departmental  and  agency  boundaries.”  Clearly,  the  time  was  ripe  for  a 
collaborative  approach  for  finding  new  solutions  to  existing  and  emerging  challenges. 

Initially  CRTI  had  14  government  departments  and  agencies  participating  in  what  was 
considered  to  be  the  first  “horizontal”  delivery  model  for  S&T  in  the  Canadian  federal 
government.  Knowledge-based  in  itself,  the  initiative  partners  detennined  investment 
priorities  according  to  a  consolidated  risk  assessment  with  both  scientific  and  intelligence 
input.  Clusters,  or  de  facto  communities  of  practice,  in  each  of  the  chemical,  biological, 
radiological-nuclear  domains  worked  together  to  identify  target  investments  and  action 
plans  for  devising  S&T  in  support  of  the  prevention  of  and  response  to  terrorist  attacks. 
The  program  sought  to  deliver  S&T  solutions  to  the  responder  community  and  other 
operational  authorities,  as  well  as  to  enhance  the  national  capability  in  research  and 
development  (R&D)  and  scientific  support  in  these  risk  areas. 

The  CRTI  KM  Team  began  its  strategic  approach  by  consulting  with  the  primary 
stakeholders,  primarily  federal  scientists  and  science  managers  who  were  involved  in  the 


scientific  Clusters.  Intuitively  understanding  KM  for  what  it  could  and  must  do  for  the 
growth  of  a  new  national  community,  they  were  able  to  articulate  a  number  of  approaches 
and  requirements  for  the  new  program.  In  turn,  the  KM  Team  developed  a  strategic 
action  plan  that  would  support  the  community  in  their  objectives.  From  the  beginning,  it 
was  a  holistic  approach  that  would  facilitate  the  creation  of  both  tacit  and  explicit 
knowledge,  use  tools  and  techniques  for  capturing  and  sharing  knowledge,  and 
disseminate  this  knowledge  to  various  stakeholders  (including  scientists,  end-users  and 
the  citizens  of  Canada)  in  various  and  appropriate  ways.  Figure  1  below  illustrates  the 
high-level  KM  model  that  was  used  to  guide  the  KM  initiatives  within  CRTI. 


Insert  Figure  1  here 

Figure  1.  High-level  model  of  KM  processes  (adapted  from  Dalkir,  2005). 


The  KM  Team  knew  that  facilitating  intellectual  capital  growth  involved  the  three 
elements  of  human,  relational,  and  structural  capital.  It  was  evident  that  expertise  was 
going  to  be  developed  on  the  individual  and  team  levels  because  the  CRTI  was 
encouraging  experts  in  multiple  disciplines  to  apply  their  existing  knowledge  in  new 
areas  and  that,  in  order  to  accomplish  this,  they  were  going  to  learn  from  each  other. 
Initially,  the  KM  focus  was  not  expressly  on  facilitating  the  human  aspect  of  intellectual 
capital,  although  it  was  fully  anticipated  and  hoped  that  this  would  happen.  Indeed, 
without  this  outcome,  the  initiative  would  not  succeed  in  achieving  its  mission.  It  was 
determined,  however,  that  without  focusing  on  the  other  types  of  intellectual  capital,  the 
development  of  human  intellectual  capital  would  not  happen  to  the  required  extent. 


On  the  structural  capital  side,  for  example,  it  was  evident  that  there  was  a  need  for  a 
technology  platform  that  participants  could  use  to  exchange,  collect  and  disseminate  their 
existing  knowledge  and  knowledge  that  was  being  created  as  the  initiative  took  hold  and 


evolved.  Within  6  months,  the  InfoPort,  a  collaborative  Community  of  Practice 
application  was  launched.  There  was  also  a  need  for  a  publication  mechanism  which 
would  permit  the  creation  of  knowledge  products  that  could  not  be  published  elsewhere 
given  that  there  were  multiple  partners  working  together  under  this  umbrella  initiative. 
Soon  the  CRTI  Secretariat  was  disseminating  various  types  of  documents  primarily  in 
electronic,  but  also  paper,  formats.  And  finally,  structures,  processes  and  tools,  which 
would  often  be  behind  the  scenes,  were  required  to  support  the  overall  functions. 

However,  building  relationships  and  developing  community  would  surely  be  the  most 
important  first  step.  The  Team  knew  that  without  a  concentrated  effort  in  facilitating 
opportunities  where  new  relationships  could  be  built  or  trust  could  be  developed,  there 
would  be  no  possibility  of  moving  ahead  in  innovative  ways  to  create  S&T  knowledge 
and  products  that  would  ensure  Canadian  public  security  and  resiliency.  Therefore,  the 
KM  approach  dedicated  a  great  deal  of  effort  to  planning  and  hosting  knowledge 
exchange  events,  workshops  dedicated  to  finding  novel  solutions,  exercises  for 
experiential  and  shared  learning,  and  communications  to  multiple  stakeholders.  It  was 
after  two  years  of  the  program,  that  the  KM  Team  acknowledged  that  it  was  time  to 
measure  the  impact  of  these  activities  on  the  CRTI  program  and  went  to  work  on  a 
research  project  with  McGill  University’s  School  of  Information  Studies  to  find  a 
mechanism  to  do  this. 

INTANGIBLE  MEASUREMENT  MODEL  AND  METHOD 

In  order  to  select  the  best  fit  measurement  model  and  method,  the  research  team  first 
undertook  an  extensive  literature  review.  This  review  looked  at  both  scholarly  and 
practitioner  publications  in  order  to  identify  the  major  measurement  models  in  use,  the 
most  widely  applied  measurement  methods  and  to  compare  the  strengths  and  weaknesses 
of  each.  All  types  of  assessments  were  included  in  this  review:  quantitative,  qualitative 
and  anecdotal.  Following  the  review,  a  recommendation  was  made  as  to  the  most 
compatible  measurement  approach  to  use  for  CRTI,  together  with  suggested  adaptations 
to  best  accommodate  the  CRTI  assessment  objectives.  Next,  the  specific  performance 


indicators  were  established  and  a  data  collection  method  was  developed.  Once  the  design 
was  complete,  ah  that  remained  was  to  collect  the  data  and  analyze  the  results.  The 
literature  review  surveyed  intangible  metrics  best  practices  and  discussed  the  pros  and 
cons  of  quantitative,  qualitative  and  anecdotal  measurement  approaches. 

Literature  Survey  of  Models  to  Measure  intangibles 

Intangibles  are  96%  of  the  value  of  the  world’s  most  successful  company  - 

Microsoft.  The  rest  is  book  value  (Nash,  2004). 

The  history  of  the  measurement  of  intangible  assets  has  run  the  gamut  from  extensions  of 
purely  financial  methods  from  the  accounting  discipline  to  more  value-based  assessment 
frameworks  (Bouteiller,  2002).  In  the  disciplines  of  accounting  and  finance,  intangibles 
or  “intangible  assets”  are  generally  referred  to  claims  (or  assets)  which  have  the  potential 
to  deliver  future  benefits  (Lev,  2001).  Intangible  assets  are  more  commonly  referred  to 
as  intellectual  capital  (IC)  and  the  terms  have  become  synonymous.  Intangible  assets  do 
not  have  a  physical  or  concrete  form  but  they  are  still  capable  of  contributing  to  the  value 
produced  by  the  company  (revenues  or  profit  in  commercial  companies,  innovations, 
service  in  non-profit  companies).  Contractor  (2001)  lists  examples  of  IC  such  as 
reputation,  brands,  goodwill,  customer  loyalty,  expertise,  or  a  unique  corporate  culture. 
Intangibles  which  have  direct  influence  on  monetary  gains  are  known  as  commercial 
intangibles  e.g.,  copyrights,  brands,  patents,  franchises,  product  quality  and  value, 
reputation,  R&D,  and  so  on.  The  rest  fall  into  the  category  of  “other”  intangibles  like 
creative  employees,  innovative  workers,  highly  motivated  staff,  enhanced  morale,  etc. 
Alternatively,  generative  and  commercial  intangibles  can  be  divided  into  individual  and 
structural  intangibles.  Individual  intangibles  are  qualities  linked  directly  to  individuals 
such  as  specialized  knowledge  and  skills,  customer  loyalty  and  supplier  loyalty.  On  the 
other  hand,  structural  intangibles  are  assets  that  are  attributed  to  interpersonal  and  inter¬ 
group  relations  rather  than  attached  to  individuals.  The  most  apparent  examples  of  these 
intangibles  are  team  work  and  corporate  culture,  (Bouteiller,  2001). 


One  of  the  first  IC  models,  the  Skandia  model,  categorized  intangible  assets  as  human 
capital,  customer  capital  and  organizational  capital  (Edvinsson  and  Sullivan,  1999). 

Three  types  of  IC  were  defined  in  this  model,  namely: 

1 .  Human  capital  -  includes  employee  brainpower,  competence,  skills, 
experience  and  knowledge; 

2.  Customer  capital  -  includes  relations  and  networks  with  partners,  suppliers, 
distributors,  and  customers.  It  also  includes  the  image  of  the  organization  in 
the  market,  its  social  identity,  and  brand  equity; 

3.  Structural  capital  -  covers  every  intellectual  capital  that  can  be  owned  by  the 
organization  including  routines,  business  processes,  practices,  databases, 
systems  and  intellectual  property. 

The  Skandia  model  of  intellectual  capital  has  subsequently  evolved,  notably  the  tenn 
“social”  or  “relationship”  capital  has  gained  popularity  over  customer  capital  in  order  to 
include  non-commercial  organizations. 

Sveiby  (1997)  described  three  types  of  “invisible”  capital  that  are  quite  similar  to  the 
Skandia  categories: 

1 .  Internal  structure  -  includes  all  the  systems,  databases,  processes  and  routines  that 
support  an  organization’s  operations  and  employees  (corresponds  to  structural 
capital); 

2.  External  structure  -  includes  all  external  relations  and  networks  that  support  the 
organization’s  operations,  including  support  and  administrative  staff  (corresponds 
to  customer  capital); 

3.  Competence  -  includes  individual  experience,  knowledge,  competence,  skills  and 
ideas  (corresponds  to  human  capital). 

Both  Sveiby  and  Edvinsson  distinguish  the  different  types  of  IC  based  on  where  they  are 
located  or  where  they  can  be  found.  In  contrast,  Andreissen  and  Tissen  (2001)  proposed 
that  intellectual  capital  be  viewed  through  the  contextual  lens  of  organizational 
competencies.  They  define  the  role  of  IC  to  essentially  be  to  strengthen  the  core 
competencies  that  an  organization  would  have  has  typically  already  identified.  Al-Ali 


(2003)  argues  that  this  may  be  too  myopic  a  model  for  IC  and  notes  that  it  tends  to  focus 
on  what  the  organization  is  supposed  to  do  now  as  opposed  to  looking  at  innovation,  the 
capacity  for  the  organization  to  learn  and  adapt  (referred  to  as  absorptive  capacity  by 
Cohen  and  Levinthal  (1990).  She  describes  the  case  of  Xerox  that  decided  not  to  invest 
in  the  PC  leaving  the  field  to  Steve  Jobs  -  Xerox  focused  on  its  identified  core 
competencies  and  failed  to  recognize  an  exceptional  opportunity  to  go  beyond  these  core 
competencies. 

Al-Ali  (2003)  goes  a  bit  further  with  her  CICM  model  (Comprehensive  Intellectual 
Capital  Model).  She  argues  that  although  being  able  to  define,  recognize  and  appreciate 
intellectual  capital  is  of  great  value,  management  needs  a  comprehensive  guide  as  to  how 
to  develop  and  leverage  intellectual  capital.  The  CICM  identifies  not  only  the  location  but 
the  function  that  the  various  different  types  of  IC  perform.  In  particular,  the  function  of 
IC  in  an  operational  production  process  is  quite  different  from  that  of  an  innovation 
process.  Al-Ali  (2003)  further  advocates  grouping  the  various  forms  of  intellectual 
capital  should  be  further  grouped  into  resources,  processes,  and  products  where  the 
relation  between  them  is  made  clear  for  the  purposes  of  management,  something  that  the 
older  existing  IC  models  do  not  do.  The  CICM  classifies  IC  as: 

1 .  Knowledge  resources  -  the  human  or  organizational  knowledge  that  goes  into  the 
making  of  products  and  services  of  the  organization  and  which  supports  the 
critical  business  processes  and  operations.  This  is  a  mix  of  human  and 
organizational  capital. 

2.  Innovation  processes  -the  processes  and  networks  that  an  organization  needs  to 
enable  its  innovation  process  and  convert  ideas  and  knowledge  resources  into 
marketable  products  and  services.  For  non-commercial  organizations,  this 
encompasses  the  critical  decision  making.  This  is  a  mix  of  social  and 
organizational  capital. 

3.  Intellectual  property  -  the  technologies,  products,  processes,  methods,  software, 
publications  and  other  works  that  the  organization  has  protected  legally  and  can 
commercialize  independently  as  an  intellectual  product  to  maximize  value.  This  is 
a  mix  of  social  and  organizational  capital. 


Based  on  this  classification  the  CICM  model  manages  the  various  groups  of  intellectual 
capital  under  three  stages  of  knowledge  management,  innovation  management  and 
intellectual  property  management. 

Table  1  summarizes  the  key  features  of  these  IC  models. 


Table  1.  Comparison  of  features  of  major  intellectual  capital  models 


Skandia  Model 

(Edvinsson  and 

Sullivan) 

Sveiby  Model 

Andriessen  and 

Thissen  Model 

CICM  model 

(Al-Ali) 

Human  capital 

Internal  structure 

Core  competencies 

(organizational) 

Knowledge 

resources 

Structural  capital 

External  structure 

Innovation  process 

Customer  capital 

Competence 

(individual) 

Intellectual  property 

For  the  CRTI  assessment,  all  of  the  models  are  compatible  and  the  one  that  was  used  was 
a  combination  of  the  Skandia  and  CICM  models.  One  of  the  major  goals  of  CRTI  is  to 
promote  innovation  in  the  S&T  sector  in  the  fight  against  terrorism.  Also,  the  notion  of 
knowledge  resources  fit  quite  well  with  the  knowledge  products  that  were  to  be  assessed 
as  stemming  from  the  KM  projects. 

Bouteiller  (2001)  notes  that  the  meaning  and  perception  of  value  depend  very  much 
depends  on  whose  perspective  is  being  taken.  Abdul-Samad  and  MacMillan  (2004) 
argue  that: 

In  the  field  of  accounting  and  finance,  numerous  studies  have  been  carried  out 
linking  intangible  assets  to  company  performance  and  profit  margin.  Several 
empirical  examples  demonstrate  that  improvement  in  key  intangible  drivers 
translates  into  increased  market  value  of  commercial  entities.  These  findings  also 
suggested  that  intangible  assets  in  organizations  have  both  direct  and  indirect 
influences  over  a  company’s  value.  For  instance,  customer  loyalty  affects  other 


factors  of  business  operational  functions  like  brands,  marketing,  services, 
communications,  and  so  on.  Through  identification  of  critical  success  factors, 
intangibles  can  also  be  used  to  drive  and  enhance  organizational  business 
performance.  In  short,  intangible  assets  are  gaining  widespread  recognition  as  key 
value  drivers  of  business  performance,  (p.  3) 

Al-Ali  also  notes  that  a  different  perspective  may  be  required  to  assess  intangible  assets. 

The  confusion  about  intellectual  capital  management  as  a  field  is  due  to  a  great 
extent  to  its  multi-disciplinary  nature.  This  is  because  intellectual  capital  includes 
disparate  types  of  resources  and  assets  that  are  human  based  (brainpower, 
competence  and  skills)  information  and  data  based  (databases,  software  & 
hardware),  innovation  based  (R&D  routines,  processes  and  practices)  and  legally 
defined  property  rights  (patents,  trade  secrets,  trademarks/brands,  and  copyright). 
As  such  it  is  a  field  of  interest  and  application  to  human  resources  professionals, 
IT  ,  R&D,  business  and  marketing  managers,  IP  lawyers  and  business  consultants, 
(available  at:  http://www.ipmall.fplc.edu/hosted  resources/al-ali/IC  main.htm). 

A  combination  of  quantitative,  qualitative  and  anecdotal  measures  must  therefore  be 
included  as  components  of  a  comprehensive  IC  measurement  model.  The  complexity 
extends  to  the  specific  indicators  to  be  measured  in  this  comprehensive  IC  model.  A 
good  guideline  can  be  taken  from  the  European  Commission’s  (1999)  MEANS 
framework. 

MEANS  is  a  program  used  by  the  European  Commission  to  classify  indicators  for  the 
evaluation  of  socio-economic  programs  based  on  their  level  of  objectives. 

The  indicators  are  classified  into  five  categories:  resource,  output,  result,  specific  impact 
and  global  impact,  which  closely  resemble  the  results-based  evaluation  framework.  One 
type  of  indicator  that  is  defined  is  a  sustainable  specific  objective,  as  shown  in  Table  2. 


Table  2.  An  extract  from  the  European  Commission  (1999,  p.  29)  definitions  of 
indicators  (MEANS) 


Level  of 

objective 

Type  of 

objective 

Definition 

Example 

Sustainable 

Specific  impact 

Sustainable  effect  as 

Results  of  output  -  a 

specific 

consequence  beyond 

measurable  result  that 

objective 

(as  opposed  to 

its  direct  short-tenn 

attests  to  the  attainment  of 

more  general 

effect  -  an  objective 

the  sustainable  objective 

(e.g.  to  create 

and  maintain  a 

outcomes) 

that  is  expected  to 

maintained  as 

high-level  of 

opposed  to  being  a 

(e.g.  number  of  new 

collaboration 

between 

different  units) 

one-shot  endeavor 

customers  through  phone 

calls  over  a  period  of  x 

months,  increased 

throughput  per  employee  for 

the  next  3  years,  etc.) 

The  MEANS  framework  includes  indicators  for  sustainable  objectives  and  effects  which 
is  an  excellent  fit  for  the  objectives  and  expected  effects  of  the  CRTI  program.  All  KM 
initiatives  should  include  an  objective  and  an  indicator  of  sustainability.  In  the  case  of 
CRTI,  the  objectives  were: 

•  To  create  clusters  of  S&T  labs; 

•  To  create  an  S&T  fund  to  build  S&T  capability; 

•  To  accelerate  technology  development  into  the  hands  of  the  first  responder 
community; 

•  To  address  any  gaps  in  S&T  capacity;  and, 

•  To  increase  horizontal  coordination  across  Government  departments  and 
agencies. 

The  subsequent  CRTI  KM  objectives  consisted  of: 


•  Building  relationships  and  organizational  structure,  processes  and  opportunities  to 
encourage  a  knowledge  culture; 

•  Collecting  and  organizing  programmatic  and  scientific  knowledge; 

•  Facilitating  the  creation  and  dissemination  of  CBRN  knowledge  including 
methods  for  data,  information  and  knowledge  exchange; 

•  Proving,  evaluating  and  adjusting  KM  activities. 

Both  the  overall  CRTI  and  the  KM  objectives  address  sustainable  goals.  This  in  turn 
means  sustainability  needs  to  be  assessed.  The  MEANS  framework  includes  such  an 
objective  in  a  results-based  assessment  framework  and  was  therefore  included  in  the 
CRTI  assessment. 

Finally,  the  assessment  of  intellectual  capital  in  R&D  or  innovation  projects  was 
surveyed  in  order  to  identify  the  best  possible  framework  and  indicators  (Subramaniam 
and  Youndt,  2005).  In  R&D,  human  capital  plays  a  significant  role  in  triggering 
innovation  and  performance.  Satisfied  and  highly  educated  knowledge  workers  tend  to 
improve  organizational  capital  (e.g.,  process,  culture,  and  brand  value)  which  is  owned 
by  organizations  and  accumulated  for  a  long-time.  Human  capital  is  only  “rented”  by  the 
organization.  Thus,  organizations’  efforts  focus  on  encouraging  employees  to 
concentrate  on  their  job  as  well  as  on  providing  them  satisfaction  to  prevent  them  from 
leaving  the  organization  or  retiring  (Pike  et  al,  2005). 

In  addition  to  reviewing  the  major  measurement  models,  the  literature  review  also 
surveyed  the  major  measurement  methods  such  as  EVA  (Economic  Value  Added),  value 
chain  scorecard,  Balanced  Scorecard,  Skandia  navigator,  Intangible  Assets  Monitor,  and 
the  Results-Based  Management  and  Accountability  approach.  Each  model  was  reviewed 
and  assessed  according  to  its  applicability  to  this  specific  KM  approach  and  to  the 
objectives  of  the  impact  assessment. 

Selection  of  the  Results-Based  Measurement  Model 

Following  this  review  the  Results-based  Management  Accountability  Framework 
(RMAF)  was  selected  (Treasury  Board  Secretariat,  2005).  RMAF  is  a  tool  designed 
specifically  to  help  track,  manage  and  measure  the  outputs  and  outcomes  of  an 


organizational  initiative  and  is  ideally  suited  for  ensuring  that  strategy  is  continuously 
aligned  with  the  objectives  and  goals  of  the  organization.  The  selected  RMAF  evaluation 
was  chosen  because  of  its  heavy  focus  on  the  cause  and  effect  relationships.  Using  the 
RMAF  helped  to  characterize  the  impact  that  the  two  major  CRTI  KM  initiatives, 
collaboration  support  and  the  InfoPort,  were  having  on  CRTI’s  broader  goals  of 
increasing  its  preparedness,  capability  and  capacity  to  respond  to  CBRN  terrorist  attacks. 
The  creation  of  the  logic  model  was  the  critical  component  that  allows  the  RMAF  to 
maintain  accurate  and  measurable  links  between  the  activities  and  their  outcomes.  The 
logic  model  graphically  shows  the  chain  of  results  between  the  activities  and  the  final 
outcomes  and  identifies  the  steps  in  between  that  must  occur  for  the  achievement  of  the 
final  outcomes.  Designing  the  logic  model  was  an  iterative  process,  the  results  of  which 
represent  a  shared  understanding  between  management,  stakeholders,  and  the  eventual 
evaluators  of  the  initiative’s  activities,  outputs  and  most  importantly,  their  outcomes. 
Designing  these  metrics  then,  consisted  of  moving  step  by  step  through  the  various  levels 
of  the  logic  model  and  identifying  the  most  appropriate  and  relevant  measurements  for 
each  output/outcome.  Figure  2  shows  the  logic  model  developed  for  one  of  the  KM 
objectives,  the  knowledge  repository  “InfoPort.” 


Insert  Figure  2  here 

Figure  2.  Logic  model  of  CRTI  InfoPort 

Next  the  team  was  able  to  identify  metrics  that  would  measure  each  output  and  outcome. 
In  this  step,  the  final  or  ultimate  CRTI  objectives  were  identified  (what  impact  is 
expected  on  the  organization?)  together  with  the  activities  and  outputs  and  outcomes. 

The  flow  diagram  can  be  derived  forwards  from  outputs  or  backwards  from  final 
objectives.  Plot  models  were  used  to  verify  the  logic.  Finally,  indicators  were  chosen  for 
each  outcome  level. 


DATA  COLLECTION,  ANALYSIS  AND  RESULTS 


A  combination  of  quantitative,  qualitative  and  anecdotal  data  was  then  collected.  As 
Mouritsen  (2001)  argues,  “intellectual  capital  is  “no  ordinary  accounting  concept””  (p. 
760).  He  strongly  advocated  a  mixed  approach  to  measuring  and  reporting  IC,  one  in 
which 

. .  .intellectual  capital  activities  (need  to)  be  related  to  narratives  of  innovation,  the 
information  society  and  ‘we-live-from-knowledge  claims...  to  create  a  persuasive 
intellectual  capital  statement  . . .  (that)  consists  not  merely  of  numbers,  but  also  of 
stories/narratives  and  visualization/sketches  that  allows  a  series  of  translations  to 
take  place,  (pp.  760-761). 

The  RMAF  measurement  was  as  comprehensive  as  possible  (Dalkir,  et  ah,  2007). 
Quantitative,  qualitative  and  anecdotal  data  were  integrated  in  order  to  provide  a  more 
complete  portrait  of  how  well  KM  objectives  were  met.  Quantitative  measures  were 
included  as  typical  multiple  choice  questions,  ranging  from  yes/no  answers  to  Likert- 
scale  multiple  choices  to  assess  attitudes.  Qualitative  data  was  gathered  both  in  the 
interviews  and  in  the  short-answer  section  of  the  data  collection  survey.  Anecdotes  were 
also  included  as  data.  In  their  book  Working  Knowledge  (1998)  Davenport  and  Prusak 
discuss  a  concept  that  they  have  coined  as  “serious  anecdote  management.”  This  concept 
refers  to  knowledge  that  is  often  passed  through  stories.  A  “serious  anecdote”  is  a  story 
that  has  a  “lessons  learned”  punch  line.  These  anecdotes  are  excellent  for  capturing  the 
context  of  a  valuable  piece  of  knowledge  and  providing  a  memorable  medium  in  which  to 
transfer  that  knowledge.  Often,  the  punch  line  carries  a  quantitative  measurement  that  can 
then  have  more  meaning  for  employees  because  it  is  delivered  within  the  context  of  a 
story  that  they  can  easily  relate  to.  Serious  anecdotes  also  illustrate  the  value  of  KM  and 
as  such  are  useful  metrics  to  include  when  evaluating  certain  initiatives  (Wiseman,  et  ah, 
2005). 


Next,  a  series  of  stakeholder  interviews  and  auditing  of  internal  documentation  was 
completed  in  order  to  clearly  identify  the  expected  outcomes  for  the  CRTI  assessment.  In 


consultation  with  the  KM  Team,  it  was  detennined  that  five  activities  would  be  selected 
for  the  measurement  framework: 

•  Collaboration  support  -  to  support  existing  and  new  communities  of  practice  in 
achieving  their  objectives;  to  enable  effective  both  within  the  community,  with 
other  communities  ,  and  with  the  wider  CRTI  community;  to  create  synergy  and 
trust;  to  create  opportunities  for  learning  and  creating  knowledge  from  CRTI 
experiences;  and  to  encourage  reciprocal  dissemination  of  knowledge  between 
CRTI  stakeholders  and  First  Responders; 

•  Intranet  portal  solutions  (IPS)  -  to  offer  centralized  online  access  to  CRTI 
information,  CBRN  expertise  and  knowledge,  and  to  encourage  virtual  exchange 
by  creating  tools  and  methods  for  sharing  knowledge; 

•  Knowledge  products  -  to  capture,  create  and  disseminate  knowledge  products  that 
apply  CBRN  knowledge  to  help  attain  ;  to  ensure  that  this  knowledge  is 
synthesized  and  packaged  in  a  manner  that  meets  the  needs  of  each  target 
audience.; 

•  Knowledge  and  information  management  structures  -  to  create  and  implement 
processes  and  tools  to  assure  CBRN  and  CRTI  knowledge  are  accessible  now  and 
in  the  future,  and 

•  Communications  and  media  relations  -  to  establish  guidelines  to  help  ensure 
effective  communications  among  CRTI  stakeholder  groups  and  with  the  media;  to 
develop  and  maintain  positive  relationships;  to  increase  CRTI’s  visibility;  and  to 
help  promote  CRTI  as  a  legitimate  and  credible  voice  on  CBRN  issues. 

The  CRTI  objectives  address  both  social  and  organizational  capital.  Although  the  value 
added  to  human  capital  was  not  directly  measured,  it  was  expected  that  increased  value  in 


structural  and  relationship  capital  would  in  turn  contribute  to  enhanced  human  capital 
(Nahapietand  Ghoshal,  1998). 

The  next  step  involved  the  formulation  of  an  appropriate  data  gathering  strategy.  It  was 
decided  that  the  most  added-value  method  would  be  to  design  an  electronic  qualitative 
survey,  using  Likert  scales  and  open  ended  questions,  which  would  be  distributed  to 
CRTI  members  through  CRTI’s  InfoPort  and  then  followed  up  with  interviews  to  gather 
additional  anecdotal  information  on  the  impact  that  KM  has  had  on  CRTI’s  ultimate 
goals.  The  survey  was  run  for  two  months  in  order  to  maximize  the  response  rate  as 
much  as  possible.  The  consisted  of  32  questions,  half  of  which  were  multiple-choice 
(Likert  scale)  and  the  other  half  were  open-ended  questions.  In  addition,  structured 
interviews  were  conducted  with  members  of  the  CRTI  KM  team  and  volunteers  in  order 
to  validate  the  RMAF  logic  models,  the  metrics  to  be  used  and  the  questionnaire  design. 
The  survey  was  initially  emailed  to  213  members.  Of  these,  129  were  actually  reached 
and  26  completed  the  survey,  resulting  in  a  response  rate  of  20%. 


RESULTS 

The  results  obtained  using  the  RMAF  model  and  method  served  to  “quantify”  and  render 
more  concrete  the  value  of  the  not-so-visible  knowledge  assets.  For  example,  the  team 
was  able  to  show  that  CRTI  KM  initiatives  aimed  at  supporting  collaboration  were  highly 
successful.  A  majority  of  respondents  felt  that  CRTI  workshops,  conferences  and 
exercises  had  increased  the  communication  of  CRTI  CBRN  information  and 
documentation,  permitted  them  to  expand  their  personal  network  or  partners  and  afforded 
valuable  learning  opportunities  for  them.  A  number  of  new  ideas,  project  or  papers  were 
the  direct  result  of  CRTI  collaboration  support  activities  and  suggestions  for 
improvement  included  encouraging  greater  audience  participation  and  providing  more 
cluster-specific  follow  up  activities. 


These  findings  indicate  that  CRTI  collaboration  support  has  helped  increase  trust  and 
synergy  within  the  CRTI  community,  helped  ensure  more  valuable  knowledge  was 
exploited,  that  collaboration  and  communication  had  increased  and  the  organizational 
memory  has  grownv. 

The  results  for  the  CRTI  InfoPort  were  more  mixed.  Table  3  below  shows  the  RMAF 
used  for  the  InfoPort. 


Table  3.  InfoPort  RMAF 


Activity 

Output 

Immediate 

outcomes 

Intermediate 

outcomes 

Final  outcomes 
(impact) 

ISP1 

Develop,  manage 

■  CRTI 

Centralized  and 

Exploitation  of 

CBRN  S&T 

and  facilitate 

InfoPort 

timely  access  to 

valuable 

knowledge  and 

usage  of  virtual 

■  Portal 

CBRN  S&T 

knowledge  is 

expertise  in  support 

workspace  for 

maintenance 

knowledge  is 

increased 

of  operations  is 

exchange  of 

,  training, 

provided  (on 

■  Increased 

developed, 

knowledge 

etc. 

demand,  as 

use/re-use 

managed  and 

between  cluster 

■  Automated 

needed) 

of  available 

leveraged 

members,  across 

profiles, 

knowledge 

clusters  and  with 

Push/pull 

Available  CBRN 

Horizontal 

other  CRTI 

disseminatio 

knowledge  is 

Awareness  of 

capability,  links 

stakeholders 

n,  filtering 

current  and  vetted 

existing 

within  CRTI 

and  alerting 

knowledge  and 

communities  are 

(automatic 

Communication 

expertise  at 

built 

and 

and  dissemination 

CRTI  is 

bulletins) 

of  CRTI 

increased 

Capability  and 

information  and 

capacity  to  respond 

ISP  2 

Collect  and 

■  Electronic 

documentation  is 

Exchange  of 

is  increased 

disseminate  CRTI 

repository  of 

enhanced 

knowledge 

(operational 

documentation 

internal 

(tacit  and 

readiness) 

(internally 

documentati 

Virtual 

explicit)  is 

■  Performance  is 

generated) 

on 

knowledge 

increased 

improved 

electronically 

sharing  and 

■  Improved 

collaboration  is 

Collaboration 

skills  or 

ISP  3 

Create  an 

■  Electronic 

enabled 

and 

competencies 

repository  of 

repository  of 

communication 

■  Improved 

relevant  external 

external 

Assistance  in 

within  and 

decision¬ 

knowledge 

CBRN 

connecting 

between 

making 

products 

documents 

members  with 

clusters  is 

■  Enhanced 

and  sources 

experts  is 

increased 

S&T  advice 

provided 

and  services 

ISP  4 

Develop  expertise 

■  Expertise 

Input  to  the 

Organizational 

provided 

locator  system 

directory 

creation  of  a 

memory  is 

(see  also  C4: 

secure,  collective 

preserved 

competency 

lab  management 

map) 

system  is 

Knowledge 

■  Expertise 

provided 

base  is 

locator 

perceived  as 

system 

complete 

ISP  5 

Develop  dynamic 

■  Lessons 

lessons  learned 

learned 

system 

database 

ISP  6 

Explore  secure 

■  Options  for 

web  processing 

secure  web 

options 

processing 
for  clusters 

(PSEPC 

pilot) 

Knowledge 

management 

project: 

Indicators 

(Quantitative) 

(Qualitative) 

ISP.  Intranet  portal 

■  Number  of  unique  visitors, 

■  Perceived  value  of  portal  and 

solutions  -  CRTI 

percentage  of  total  using  system 

knowledge  objects 

InfoPort  &  other 

(trend) 

■  Frequency  of  use 

online  tools 

■  Number  of  hits,  downloads  on 

■  Time  to  access 

portal,  dwell  time  (trend) 

information/knowledge 

■  Searching  precision  and  recall  /  time 

■  Number/examples  of  occurrences 

to  find  object 

where  access  to  knowledge  from 

■  Number  of  experts  in  directory  / 

portal,  expertise  directory  or  other 

domains  covered 

databases  resulted  in  acquiring  a 

■  Time  required  to  find  expert 

new  skill  or  competence,  an 

■  Number  of  contacts  /  relationships 

improvement  in  quality  or 

made  through  portal  (directly  or 

efficiency,  or  solving  a  problem 

indirectly)  -  (SNA) 

■  Estimate  of  time  or  cost  avoided 

■  Number  of  referrals  made 

by  leveraging  expert  knowledge 

■  Number/range  of  knowledge  objects 

or  knowledge  base  (also,  reduced 

available  on  portal 

learning  curve,  reduced  training) 

■  Number  of  contributions  made  to 

■  Improvement  in  awareness  of 

the  portal  /  knowledge  base 

available  information 

■  Number  /  range  of  lessons  learned  in 

■  End-user  satisfaction  /  increased 

database 

ability  to  work 

■  Number  of  alerts  sent  out 

■  Perception  of  confidence 

■  Number  of  alerts  made  use  of 

■  Frequency  of  contributions  / 

Contribution  rate  increase  /  decrease 

A  summary  of  the  results  for  each  of  the  InfoPort  indicators,  as  obtained  from  the  survey, 
are  presented  in  Table  4. 


Table  4.  InfoPort  questions  and  results  (extracted  from  survey). 


No. 

Question  Text 

Response 

1 

Are  you  able  to  find  information  on  the  InfoPort  in  a 

reasonable  amount  of  time? 

68%  Yes 

3 

Does  the  InfoPort’ s  search  function  meet  your  needs? 

68%  Yes 

5 

Do  you  feel  you  have  centralized  access  to  CBRN  S&T 

knowledge  through  the  InfoPort? 

68%  Yes 

7 

Have  you  accessed  workshop  and  conference  proceeding 

summaries  from  the  InfoPort? 

32%  Yes 

9 

Do  you  feel  the  information  available  on  the  InfoPort  is 

up  to  date? 

79.2%  Yes 

11 

Have  you  filled  in  your  user  profile  on  the  CRTI 

InfoPort? 

73.9%  Yes 

13 

Can  you  locate  an  expert  through  the  InfoPort  when  you 

need  one? 

65%  Yes 

15 

Have  you  ever  been  contacted  as  an  expert  through  your 

profile  on  the  InfoPort? 

0%  Yes 

17 

Has  the  CRTI  InfoPort  helped  increase  your  awareness  of 

CRTI  expertise,  projects,  gaps  and  successes? 

52.2%  Yes 

19 

How  often  do  you  use  the  information/knowledge 

available  through  the  InfoPort  to  accomplish  a  task? 

18.2%  never 

54.5%  rarely 

27.3%  sometimes 

20 

The  information  on  the  CRTI  InfoPort  represents  the 

best/most  complete  infonnation  that  you  need  for  your 

job. 

4.5%  strongly  disagree 

27.3%  disagree 

59.1%  no  opinion 

9.1%  agree 

25 

How  frequently  do  you  look  at  “alerts”  or  notices  of  new 

9.1%  never 

additions  to  InfoPort? 

36.4%  rarely 

22.7%  sometimes 

22.7%  often 

9.1%  very  often 

In  summary,  with  respect  to  the  InfoPort,  %  of  participants  had  filled  out  their  online  user 
profile  but  others  stated  they  were  reticent  due  to  issues  with  unsolicited  emails  and 
general  privacy.  While  the  majority  felt  the  content  was  easy  to  access  and  up  to  date, 
there  were  a  number  of  qualifiers  to  be  found  in  the  qualitative  data  collected,  both  in  the 
survey  short  answers  and  in  the  interviews.  Participants  expressed  some  concerns  with 
the  user-friendliness  of  the  system  and  the  fact  that  the  content  needed  to  be  much  more 
complete  and  updated  more  frequently  in  order  to  serve  a  more  operational  support 
function.  The  greatest  gap  was  that  not  a  single  person  reported  locating  an  expert 
through  the  InfoPort.  This  appears  to  be  due  to  the  large  number  of  competing 
information  sources,  most  of  which  are  more  mature,  better  maintained  due  to  more 
abundant  resources  and  almost  all  were  external  links  (international).  Another  issue  with 
expertise  location  was  that  the  level  of  specificity  was  not  sufficient  to  enable  users  to 
locate  the  specialized,  technical  expertise  they  would  require  to  fulfill  their 
responsibilities.  The  top  content  picks  for  the  InfoPort  were 

-  Lessons  learned  reports  from  exercises; 

-  Descriptions  of  cluster  activities  and  projects  in  progress;  and, 

-  “As  was  said”  summaries. 

It  is  interesting  to  note  that  these  three  types  of  content  represent  value  added  content: 
that  is  to  say,  they  were  not  simply  cut  and  pasted  into  the  InfoPort  as  is  from  another 
source  but  the  CRTI  team  had  contributed  to  the  creation  of  new  knowledge  and  to  the 
contextualization  of  existing  knowledge.  There  is  clearly  a  value  perceived  by  the 
InfoPort  clients  when  the  original  content  has  been  enhanced  in  this  way. 


The  major  suggestions  to  improve  the  InfoPort  included: 


•  Make  it  easier  and  more  efficient  to  use 

•  Organize  the  content  better; 

•  Make  infonnation  with  a  short  shelf  life  available  much  sooner  while  posting 
more  scientific  content  at  a  later  date; 

•  Broaden  the  scope  of  content  -  more  technical,  more  international,  more 
connected  to  external  sites;  and, 

•  The  expert  directory  and  its  objectives  need  to  be  revisited. 

As  with  the  collaboration  support  project,  the  InfoPort  also  contributes  to  the  final  or 
impact  outcomes  of  developing,  managing  and  leveraging  CBRN  S&T  knowledge  in 
support  of  operations,  building  horizontal  capability  and  links  with  the  CRTI 
communities. 

Only  anecdotal  data  was  collected  to  support  the  final  outcome  of  increased  capability 
and  capacity  to  respond: 

•  Improved  performance; 

•  Improved  skills  or  competencies; 

•  Improved  decision-making; 

•  Enhanced  S&T  advice  and  services  provided. 


SOLUTIONS  AND  RECOMMENDATIONS 

The  overriding  result  from  the  InfoPort  evaluation  was  that  there  was  a  strong  preference  for 
person-to-person  contact  within  and  between  CRTI  clusters,  as  well  as  with  external  networks. 
This  result  is  not  surprising  given  the  effort  devoted  to  creating  the  clusters  as  vibrant 
communities  of  practice  in  the  first  place.  However,  the  results  do  indicate  that  technology  still 
falls  short  in  providing  a  virtual  conduit  for  knowledge  sharing  interactions.  The  measurement 
model  and  method  selected  proved  to  be  a  good  fit  for  the  CRTI  objectives  as  there  was  a  strong 
case  for  “contribution”  if  not  outright  causality.  What  is  even  more  important,  however,  was  the 
development  of  the  correct  indicators.  As  is  the  case  with  every  intangible  assessment  approach, 


it  is  crucial  to  customize  the  metrics  to  fit  the  organizational  objectives  at  hand.  The  research 
team  found  that  at  least  three  months  were  required  for  the  stakeholder  interviews,  determination 
of  measurable  objectives  and  the  customization  of  the  RMAF  method.  While  this  is  a  substantial 
up-front  cost  in  terms  of  time  and  effort,  the  end  results  are  certainly  worth  it. 

In  the  case  of  CRTI,  the  RMAF  represented  a  summative  assessment  of  specific  KM 
contributions  to  CRTI  goals  at  the  five-year  stage  of  the  CRTI  program.  What  is  interesting  is 
that  now  the  model  and  the  indicators  have  been  developed  and  validated,  the  approach  is  easily 
re-applied  as  future  formative  measures  (for  example,  on  a  yearly  basis)  to  better  monitor 
progress  towards  organizational  goals.  At  the  same  time,  the  measurement  model  can  be  used  to 
ensure  a  continually  re-aligned  model  with  respect  to  evolving  organizational  goals. 

This  particular  approach  should  be  easily  applicable  to  other  research-based  organizations,  S&T 
government  programs  and  any  organization  that  needs  to  address  a  sustainable  innovation  goal  as 
part  of  its  core  mission. 


FUTURE  RESEARCH  DIRECTIONS 

The  focus  in  this  assessment  was  on  organizational  and  social  capital.  It  would  be 
interesting  to  extend  the  RMAF  to  include  indicators  of  the  value  added  to  the  human 
capital  that  can  be  harnessed  in  the  S&T  domain  to  fight  terrorism.  These  indicators 
could  be  applied  over  time  to  assess  whether  the  expected  increase  in  the  human  expertise 
and  competencies  that  can  be  harnessed  to  provide  a  response  to  terrorist  threats  would  be 
a  strong  addition  to  this  measurement  model.  Some  of  the  data  collection  questions 
could,  for  example,  include: 

•  Do  you  feel  you  are  able  to  locate  the  appropriate  person  to  help  you  out  more 
easily  than  5  years  ago? 

•  Do  you  feel  you  have  a  good  overview  of  the  type  of  human  resources  available 
to  you  through  the  CRTI  clusters?  Has  this  overview  improved  over  the  last  5 
years? 


Additional  indicators  could  also  be  added  to  measure  innovation  and  research 
productivity,  such  as  number  of  patents  over  the  5  years  studied,  number  of  publications 
and  citation  index  measures  (Mouritsen,  et  al,  2004). 

Additional  work  is  also  needed  to  triangulate  the  results  obtained  using  the  RMAF  model. 
Given  that  causality  cannot  be  rigorously  demonstrated,  the  use  of  other  valuation 
methods  such  as  the  Balanced  Scorecard  (Kaplan  and  Norton,  1997)  would  serve  to 
strengthen  the  validity  and  reliability  of  the  results  obtained. 

Finally,  a  future  research  direction  would  be  to  extend  the  results-based  management 
accountability  framework  used  to  assess  individual  CRTI  intellectual  assets  to  a  more 
holistic  or  systems-based  evaluation  framework.  Chen  et  al  (2004)  notes  that  there  is  a 
need  to  understand  the  causal  relationship  among  the  intangibles  to  be  able  to  measure 
and  monitor  them  so  as  to  steer  them  towards  the  firm’s  success.  The  systems  thinking 
approach  (Richmond,  2001;  Stemman,  2000)  is  also  an  excellent  means  of  visualizing  the 
entirety  of  intellectual  assets  and  valuing  not  only  each  individual  one  but  also  assessing 
the  value  of  their  interactions. 

Recently,  value  maps  have  been  advocated  as  holistic  measures  of  the  value  added  by 
knowledge  assets  that  are  visualized  as  “unbreakable”  wholes  or  a  gestalt  (Jhunjhunwala, 
2009).  In  value  maps,  the  perfonnance  of  each  intangible  is  linked  to  others.  Bygdas  et 
al.  (2004)  describe  an  activity-based  value  map  approach  for  measuring  IC  that  consists 
of  three  phases:  modeling,  measuring  and  action.  The  modeling  phase  begins  with  a 
mapping  and  description  of  the  company’s  critical  value  processes,  the  activities  in  those 
and  a  description  of  how  they  are  interrelated.  In  the  measuring  phase,  the  resources 
required  for  each  activity  is  mapped.  For  each  activity  there  is  a  mapping  of  what 
intellectual  (critical  and  necessary)  resources  are  needed  to  give  sufficient  quality  and 
frequency  of  the  activities.  This  approach  is  quite  compatible  with  the  result-based 
assessment  that  was  carried  out  on  CRTI  KM  activities  and  would  prove  a  useful 
extension  to  the  existing  framework. 


CONCLUSION 


Without  the  existence  of  generally  accepted  KM  measurement  tools,  in  general,  and  those  which 
would  address  impact  on  outcomes,  in  specific,  the  CRTI  KM  Team  solicited  expert  KM  advice 
and  research  to  final  a  novel  solution.  While  many  models  exist  for  measuring  or  evaluating 
intellectual  capital,  most  were  exceedingly  complex  for  a  small  organization  or  directed  at  for- 
profit  enterprises.  The  RMAF  approach  afforded  the  ability  to  evaluate  the  impact  of  KM 
activities  on  outputs  and  outcomes.  The  final  evaluation  has  permitted  the  CRTI  to  acknowledge 
the  relative  success  of  its  program  and  to  adjust  future  efforts  according  to  these  results. 

The  RMAF  also  provides  a  broader  KM  opportunity  in  that  by  using  it  as  a  dynamic  planning  and 
management  tool  from  the  onset,  the  measurement  process  will  be  facilitated  and  will  result  in  a 
natural  measurement  and  evaluation  output.  In  future  efforts,  there  will  be  a  need  to  address  all 
types  of  intellectual  capital  with  a  focus  on  the  impact  of  KM  activities  on  human  capital 
creation,  development,  and  outcomes.  The  current  analysis  indicates  that  KM  activities  do  have 
an  impact  and  it  will  be  essential  to  include  a  holistic  approach  in  measuring  intangible  assets  on 
a  continuing  basis. 

The  RMAF  measurement  model  has  also  proven  to  be  very  compatible  with  the  major  models 
(and  types)  of  intellectual  capital.  The  comprehensive,  integrated  assessment  framework  and 
guide  presented  here  should  prove  to  be  useful  to  researchers  and  practitioners  in  the  assessment 
of  intangible  value. 
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KEY  TERMS  &  DEFINITIONS  (SUBHEAD  1  STYLE) 


Intangible  Asset 

Intangible  assets  are  claims  to  future  benefits  (e.g.,  cost  savings,  increased  revenues)  that 
do  not  have  a  physical  (e.g.,  factory)  or  financial  (e.g.,  a  stock  or  a  bond)  embodiment 
(Lev,  2001). 

Knowledge  Management 

Knowledge  Management  is  the  discipline  that  systematizes  the  capture,  codification, 
sharing  and  dissemination  of  knowledge  in  order  to  leverage  individual,  group  and 
organizational  intellectual  capital  for  increased  innovation,  value  and  productivity.  ( CRTI 
McGill  KM  Team) 

Innovation  Management 

Innovation  management  is  a  tenn  used  to  refer  to  new  practices  and  tools  improve  the 
organization's  ability  to  innovate  by  creating  the  right  culture  (e.g.  soliciting  and 
encouraging  employees'  submission  of  ideas,  and  developing  new  products  and 
solutions).  (Al-Ali). 

Intellectual  Asset 

Intellectual  assets  is  the  tenn  preferred  by  intellectual  property  lawyers  and  professionals 
to  refer  to  intellectual  property  (particularly  patents,  trademarks  and  copyrights)  since 
their  value  can  be  more  accurately  perceived  or  evaluated  -  hence  the  word  "asset".  (Al- 
Ali). 


Intellectual  Capital 


Intellectual  capital  is  the  part  of  a  country's  or  a  firm’s  capital  or  an  individual’s  human 
capital  that  consists  of  ideas  rather  than  something  more  physical.  It  can  often  be 
protected  through  patents  or  other  intellectual  property  laws.  (The  Economist). 


Human  Capital 

Human  capital  is  defined  as  the  knowledge  that  employees  bring  and  take  with 
them  when  they  join  or  leave  the  firm.  It  includes  the  knowledge,  skills, 
experiences  and  abilities  of  people.  (Grasenik  and  Low,  2004) 

Structural  Capital 

Structural  capital  is  defined  as  the  pool  of  knowledge  that  remains  with  the  firm 
at  the  end  of  work,  after  employees  have  left  (Stewart,  1997).  It  comprises  the 
organisational  routines,  procedures,  systems,  cultures,  databases,  etc.  Some  of 
this  may  be  intellectual  property.  (Grasenik  and  Low,  2004) 

Relational  Capital 

Relational  capital  is  defined  as  all  resources  linked  to  the  external  relationships 
of  the  firm  such  as  customers,  suppliers  or  R&D  partners.  It  comprises  that  part 
of  human  and  structural  capital  affecting  the  firm’s  relations  with  stakeholders 
(investors,  creditors,  customers,  suppliers,  etc.)  plus  the  perceptions  that  are 
held  about  the  firm  (brand,  reputation,  etc.).  (Grasenik  and  Low,  2004) 

Outcome  Based  Evaluation 


Outcomes  Based  Evaluation  is  an  approach  to  measuring  the  effects  of  a  project  or  an 
institution's  services  and  activities  on  the  target  audience  that  these  programs  seek  to 
benefit  or  serve.  (Publishers  Bindings  Online). 


Qualitative  measurement 


Qualitative  measurement  attempts  to  provide  context  and  value  to  notions  that  are  either 
difficult  or  irrelevant  to  quantify,  such  as  the  value  an  individual  employee  gains  from 
being  a  member  of  a  community  of  practice  (Smith,  2001). 

Quantitative  measurement 

Quantitative  measurement  means  assigning  a  numerical  value  to  an  observable 
phenomenon,  such  as  the  number  of  times  an  employee  visits  a  specific  KM  web  portal. 
This  type  of  measurement  would  be  described  as  a  usage  metric  (Hall,  2000). 

Results-based  evaluation 

Results-based  management  is  a  life-cycle  approach  to  management  that  integrates 
strategy,  people,  resources,  processes  and  measurements  to  improve  decision-making, 
transparency,  and  accountability.  The  approach  focuses  on  achieving  outcomes, 
implementing  performance  measurement,  learning  and  changing,  and  reporting 
performance.  (Treasury  Board  of  Canada). 


END  NOTES 

i.  The  Chemical,  Biological,  Radiological,  Nuclear  and  Explosives  (CBRNE)  Research  and  Technology 

Initiative  Science) 

ii.  Defined  as:  the  discipline  that  systematizes  the  capture,  codification,  sharing  and  dissemination  of 

knowledge  in  order  to  leverage  individual,  group  and  organizational  intellectual  capital  for 
increased  innovation,  value  and  productivity. 

iii.  The  Initiative  originally  covered  only  CBRN.  Explosive  threats  were  added  in  2006. 

iv.  http://www2.parl.gc.ca/content/hoc/Committee/371/INST/Reports/RP1032098/indurp05/indurp05 

-e.pdf 


v.  Excerpt  from  Dalkir,  Kimiz,  Erica  Wiseman  and  Michael  Shula.  CRTI  Knowledge  Management 
Metrics  Project  Report:  a  survey  evaluation  of  major  knowledge  management  objectives. 
Submitted  February  2007. 


